FIG. 16 shows the arrangement of a conventional image sensing device.
FIG. 16 is a view showing an outline of the arrangement of a generally used CCD. This CCD primarily includes a light receiving unit 1 comprising a plurality of photoelectric conversion elements, optical black (OB) units 2L and 2R in which light-shielding members are arranged on some photoelectric conversion elements in order to prevent light incidence on these photoelectric conversion elements, a horizontal transfer CCD (HCCD) 4, a vertical transfer CCD (VCCD, not shown), and a charge-voltage conversion amplifier 5.
A driving method of this CCD is to transfer electric charge generated in the light receiving unit 1 to the VCCD, and sequentially transfer this electric charge so as to read out the charge from the HCCD 4 for each horizontal line. The output from the HCCD 4 is subjected to charge-voltage conversion by the charge-voltage conversion amplifier 105, and the voltage signal is output.
FIG. 17 is a view showing the arrangement of a general digital camera. When a user operates a camera operation switch 101 (including, e.g., a main switch and a release switch), an overall control circuit 100 detects the operation and starts supplying power to other circuit blocks.
An object image in the field angle is formed on an image sensing device 104 via main photographing optical systems 102 and 103, and converted into an electrical signal by the image sensing device 104. The output electrical signal from this image sensing device 104 is supplied to an A/D conversion circuit 106 via a CDS/AGC circuit 105, and converted into a digital signal pixel-by-pixel by the A/D conversion circuit 106. On the basis of a signal provided by a timing generator 108 which determines driving timings as a whole, a driver circuit 107 controls charge accumulation in the image sensing device 104, charge transfer in the horizontal and vertical directions, and the like. Also, the CDS/AGC circuit 105 and the A/D conversion circuit 106 operate in accordance with the timings generated by the timing generator 108.
The output image signal from the A/D conversion circuit 106 is supplied to a memory controller 115 via a selector 109 controlled by the overall control CPU 100, and written in a frame memory 116. This image signal written in the frame memory 116 is read out by the memory controller 115 and transferred to a camera digital signal processor (DSP) 110 via the selector 109. This camera DSP 110 generates R, G, and B color signals on the basis of the image signal supplied from the frame memory 116.
In a state before photography, the result of processing by the camera DSP 110 is normally transferred periodically (frame-by-frame) to a video memory 111 to display a sensed image on a monitor display 112, thereby allowing this monitor display 112 to function as an electronic finder.
On the other hand, when the user designates photography by operating the camera operation switch 101, an image signal of one frame is read out from the frame memory 116 in accordance with control by the overall control CPU 100. This image signal is processed by the camera DSP 110, and the result is temporarily written in a work memory 113. The image signal written in the work memory 113 is compressed by a predetermined compression format by a compressor-expander 114, and the result is saved in an external nonvolatile memory 117 (e.g., a flash memory).
To observe a photographed image, an image signal compressed and saved in the external nonvolatile memory 117 is expanded by the compressor-expander 114. The result is transferred to the video memory 111 and displayed on the monitor display 112.
As described above, in this general digital camera the output signal from the image sensing device 104 is processed in almost real time and output to the memory or the monitor display.
To improve the performance of continuous shot photography or the like in the above digital camera, e.g., to realize a continuous shot photography capability of about 10 frames/sec, it is necessary to improve the whole system including the image sensing device, e.g., to increase the rate of read from the image sensing device and increase the rate of write of an image signal to the frame memory or the like.
In a conventional image sensing apparatus such as a digital video camera or digital still camera using a solid-state image sensing device such as a CCD, as disclosed in, e.g., Japanese Patent Laid-Open Nos. 5-137059 and 6-141246, the resolution is increased by sensing an object divisionally by using a plurality of CCDs, and obtaining a sensed image of the whole object by synthesizing these partial images by image processing.
Recently, an image sensing system using a CCD composed of a few million pixels has been developed, so various improvements for performing signal processing at high speed have been made.
Japanese Patent Laid-Open No. 3-74991 discloses an image reading apparatus which uses a plurality of image sensing devices for divisionally reading image information, and in which the image sensing ranges of adjacent image sensing devices are overlapped in the boundary, and images sensed by these image sensing devices are pasted in the boundary to obtain an image of the whole image information. In this image reading apparatus, a pixel position having a small spatial density change with respect to a nearby pixel is detected from constituent image data. In this pixel position, image data forming an image sensed by one image sensing device and image data forming an image sensed by the other image sensing device are pasted.
However, in either apparatus in which portions of an object image are divisionally sensed by a plurality of image sensing devices and these sensed images are pasted to obtain a sensed image of the whole object as described above, if these image sensing devices have sensitivity differences, image density gap will be noticeable due to these sensitivity differences in the pasted portions (boundaries). As a consequence, an unnatural sensed image is obtained.
Also, Japanese Patent Laid-Open Nos. 5-137059 and 6-141246 mentioned above disclose the technologies of divisionally sensing an object image by a plurality of image sensing devices. However, these references do not describe the problem of density gap in the boundaries between sensed images due to the sensitivity differences between the image sensing devices, and hence do not disclose any method of solving this problem.
To obtain a sensed image having inconspicuous pasted portions, Japanese Patent Laid-Open No. 3-74991 discloses a method of improving poor image quality such as image disturbance in boundaries. Since, however, two partial images are simply pasted by detecting line-by-line a position at which a density change in the boundary is small, the pasting process is complicated and time-consuming.
In addition, this method merely controls the pasting position of two sensed images for each lateral line. Therefore, it is difficult to effectively suppress abrupt density changes in boundaries in the longitudinal direction. In particular, it is difficult to effectively reduce density gap in boundaries by sensitivity differences between image sensing devices and thereby obtain a sensed image having inconspicuous pasted portions.
As a method of improving this difficulty, Japanese Patent Laid-Open No. 11-055558 proposes a digital camera which divisionally senses the left and right portions of an object image by two CCDs arranged such that the sensed images overlap each other in the boundary. These two sensed images are stored in an image memory via an analog signal processor and an A/D converter. After that, a shading corrector corrects variations in the sensitivity distribution in the image sensing surface. Then, an image synthesizer pastes the portions in the boundary to generate a sensed image of the entire object.
In this method, an image of a boundary portion is generated by reducing a density difference in the boundary portion by performing processing such as average value calculation. A sensed image is generated by synthesizing this boundary image and the left and right images excluding the boundary image. However, an optical image of an object must be supplied to a plurality of image sensing devices after being divided into a plurality of images partly overlapping each other. A three-dimensional space required by this method increases the size of the apparatus, requires high assembly accuracy, and consequently increases the manufacturing cost.
FIG. 18 is a view schematically showing the device structure of a two-output-type CCD image sensing device obtained by splitting a horizontal transfer CCD.
In this image sensing device shown in FIG. 18, electric charge generated pixel-by-pixel in a photodiode unit 59 is transferred to a vertical transfer CCD (VCCD) 60 at once at a predetermined timing. At the next timing, this electric charge in the VCCD 60 is transferred line-by-line to left and right horizontal transfer CCDs (HCCDs) 57 and 58. The HCCD 57 transfers this electric charge to a left amplifier 55 for each transfer clock, and the HCCD 58 transfers the electric charge to a right amplifier 56 for each transfer clock. In this image sensing device, therefore, a sensed image signal is read out as it is divided into two, left and right regions on the two sides of the center of the frame. Although not shown, optical black (OB) units shielded from light by, e.g., aluminum are present on the left and right ends of an effective pixel area which, among other pixels, generates electric charge when exposed.
FIG. 19 is a block diagram of a signal processing circuit for processing an output signal from an image sensing device of the type shown in FIG. 18. Left and right image signals sensed by a CCD 11 are subjected to CDS/AGC processing by two CDS/AGC circuits 14 and 15 and converted into digital signals by two A/D conversion circuits 16 and 17, respectively. These digital signals are stored in frame memories 20 and 23.
Since the image sensing device 11 and the CDS/AGC circuits 14 and 15 are AC-coupled, DC components are removed when image signals are input from the image sensing device 11 to the CDS/AGC circuits 14 and 15. To regenerate these DC components, clamping circuits 18 and 19 clamp the image signals sensed in the left and right regions of the image sensing device 11 in accordance with the pixel values of Optical Black (OB) pixel portions in these regions. That is, the levels of the whole image signals sensed in the left and right regions of the image sensing device 11 are adjusted such that the pixel values of these OB pixel portions have a predetermined level.
As described above, the OB pixel portions of the image sensing device 11 are positioned on the left and right ends of the effective pixel area. The pixel values of these OB pixel portions are slightly different owing to shading of a dark current and the like. Accordingly, when the frame is divided into two, left and right regions and DC levels are regenerated by separately clamping image signals in these left and right regions on the basis of the pixel values of the OB pixel portions, an offset difference is produced between the left and right regions of the frame. Consequently, an image having different DC levels corresponding to the level difference between the pixel values of the two OB pixel portions is obtained.
The left and right image signals thus processed separately are synthesized by an image synthesizing circuit 24. A color processor 25 performs color processing such as color interpolation and gamma conversion for the synthetic image signal, thereby forming an image signal of one image.
This technology of equipping an image sensing device with a plurality of output terminals and simultaneously reading out image signals from these output terminals is an essential technology of allowing future digital cameras to approach or exceed the performance of silver halide cameras (a single-lens reflex camera capable of photographing about 8 frames/sec is already realized as a product).
This technology of equipping an image sensing device with a plurality of output terminals is advantageous in speed. However, the technology is obviously more disadvantageous than a single-output image sensing device from a viewpoint of output level matching.
That is, to regenerate a DC component removed when an image signal is supplied to the CDS/AGC circuit, the CDS/AGC circuit or the camera DSP conventionally performs a clamping process in each region of an image sensing device on the basis of the optical black (OB) level. In this method, however, if there is a difference between the OB level for determining the clamp level and the black level of an effective pixel area, or if OB levels output from a plurality of output terminals of the image sensing device are different owing to dark current shading, offset differences between the outputs from the plurality of regions of the image sensing device cannot be completely removed. Consequently, boundaries between the plurality of regions appear in the synthetic image.
Accordingly, it is an important subject to remove offset differences between output image signals from a plurality of output terminals of an image sensing device.
This will be explained in more detail with reference to FIGS. 20A and 20B.
FIG. 20A shows the arrangement of another conventional image sensing device.
That is, FIG. 20A is a view showing an arrangement for dividing a photoelectric conversion output from a light receiving unit 1 to two, left and right horizontal transfer CCDs (HCCDs) 4L and 4R and simultaneously reading out these outputs. In this image sensing device having the configuration shown in FIG. 20A, photoelectric conversion outputs are transferred in the form of electric charge by the HCCDs 4L and 4R for each horizontal line. More specifically, the left-half and right-half photoelectric conversion outputs are transferred in the form of electric charge to the left and right by the HCCDs 4L and 4R, respectively, and subjected to charge-voltage conversion by discretely prepared charge-voltage conversion amplifiers 5L and 5R.
In this configuration, read operations can be performed parallel by the two HCCDs 4L and 4R. So, one frame can be read out in a driving time half that of the configuration shown in FIG. 16.
Additionally, in the arrangement shown in FIG. 20A, the image sensing unit is realized by one CCD. Since means such as an optical path divider is unnecessary, it is possible to simplify the arrangement of the image sensing system and reduce the cost of the system. However, the left and right regions of a sensed image are separately read out by the independent systems (i.e., the HCCD 4L and the amplifier 5L, and the HCCD 4R and the amplifier 5R). So, a difference in the boundary is conspicuous.
FIG. 20B shows examples of photoelectric conversion outputs of one horizontal line indicated by a–a′ in FIG. 20A.
FIG. 20A shows an example of a landscape on a clear day in which the sun, mountains, trees, and grass are shown. Referring to FIG. 20B, a level A indicates an output from an OB portion 2R in FIG. 20A, a level B indicates an output from an OB portion 2L, and a level C indicates an output from a portion corresponding to the sun. A difference D is a level difference produced when the image is read out by the left and right independent systems described above.
A method of making this level difference be naturally seen in the frame is called frame matching. One example is a method of making a density difference in a boundary inconspicuous. This method is to correct the density difference in the boundary on the basis of the degree of correlation obtained from image data in the boundary. Improving the accuracy of this method requires a countermeasure against a dark current which lowers the S/N.